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Abstract 

We present a novel coalgebraic formulation of infinite extensive games. We define both 
the game trees and the strategy profiles by possibly infinite systems of corecursive equations. 
Certain strategy profiles are proved to be subgame perfect equilibria using a novel proof 
principle of predicate coinduction which is shown to be sound by reducing it to Kozen's 
metric coinduction. We characterize all subgame perfect equilibria for the dollar auction 
game. The economically interesting feature is that in order to prove these results we do not 
need to rely on continuity assumptions on the payoffs which amount to discounting the future. 
In particular, we prove a form of one-deviation principle without any such assumptions. This 
suggests that coalgebra supports a more adequate treatment of infinite-horizon models in 
game theory and economics. 

1 Introduction 

Infinite structures turn up at many places in economic theory. Prominent examples are infinite 
games. The standard approach is to employ discounted dynamic programming to derive equilibria 
[2J [TU] . In this paper we take a different approach. We shall use the methods of coalgebra [TU [5] 
to analyze two basic examples of infinite games without resorting to discounting and induction. 

We use coalgebraic methods to define infinite extensive games and strategy profiles, and the 
notion of subgame perfect equilibrium. We then prove that certain profiles are subgame 
perfect equilibria. This leads us to formulate an (apparently) novel notion of predicate coinduction, 
which is shown to be sound by reducing it to Kozen's metric coinduction [7] . We give a complete 
characterization of the subgame perfect equilibria of the dollar auction game. We also prove a 
version of a standard game theoretical result, the one deviation principle, without needing 
future discounting assumptions. 

Our work is inspired by recent work by Pierre Lescanne [SJ |5], but it is explicitly coalge- 
braic, leading to a mathematically richer and more general approach. The coalgebraic approach 
we present is promising for further applications to economic modeling and we will discuss some 
directions for future work in the last section. 



2 Games 

We shall assume familiarity with the basic elements of coalgebra [HJ [5] . 

We consider possibly infinite games of perfect information in extensive form and fix the fol- 
lowing sets: 
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• A set A of agents or players. In our examples, we confine ourselves to two-player games, 
with A = {A,B}. 

• A set C of choices. We restrict our discussion to games where a player has a choice of 
two options, left or right, at every stage in the game where it is their turn to play. Thus 
C = {\,r}. 

• A set U of utility functions, which assign a utility to each player. We take U = R A . 
The set of game trees is defined to be (the carrier of) the final coalgebra (Q, 7) of the functor 

F Q : X ^ U + Ax X x X. 

on the category of Set. The game tree is a possibly infinite binary tree. The binary nodes are 
labelled with an agent whose turn it is to play at that stage in the game. The nodes have the form 

(a,g\,g r ), 

where a is the agent label, and g\, g r are the subgames corresponding to the left and right choices 
respectively. The leaf nodes are labelled with utility functions, representing the pay off for the 
game. 

2.1 The 0/1 game 

We define the utility functions to be 

v := [A i-t 0, fii-> 1], to := [i h> 1, Bh>0]. 
The game is defined by the following pair of mutually corecursive equations: 

G = (A,v,H) 
H = (B,w,G). 

We can depict the game tree as follows: 



w 



w 



More formally, the above equations define a Fc-coalgebra a : {G,H} — > Fg{G,H} on the set 
{G,H}. The 0/1 game is given by [G], the image of G under the unique coalgebra morphism 

[■]:({G, 7) 

from ({G,H},a) to the final coalgebra. 

Note that we can read the utility function v as l A loses and B wins', while w corresponds to 
l A wins and B loses'. Thus the player who chooses to stop first loses. 

2.2 The dollar auction 

The 0/1 game has only finite states; it can be represented by a finite system of equations. The 
dollar auction game, a well-known example in game theory, is an infinite-state refinement, where 
the utility functions change as we move down the tree. 

We fix a real number r, and define utility functions v n , w n for each stage n G N: 



[A 1 y -n, B H> r - n], 



Wr, 



[A h- > r — (n + 1), B 1— > -n). 
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We define a game by simultaneous corecursion on the infinite set of variables {G n , H n \ n G N}: 



G n = (A,v n ,H n ) 
H n = (B,w n ,G n+ i). 

The dollar auction game is again given by [Go], the image of Go under the unique coalgebra 
morphism into the final coalgebra. We can depict the game tree as follows: 



"'0 



The motivation behind this game is as follows [15]. The value of the asset being bid for is r; 
in the original example, r is one dollar or 100 cents. The asset goes to the highest bidder, who is 
left with a profit of r — b, where b is the value of his highest bid. The loser must also pay the value 
of his highest bid, while getting nothing in return. The above definition describes the situation 
where each player raises their bid by one cent at each stage in the game. A player either gives 
up and finishes the game, conceding the auction to the other player and accepting their loss, or 
continues, hoping that the other player will give up. Both players have an incentive to continue 
playing well beyond the point where both will make a loss, in order to try to minimize their losses. 



3 Strategy profiles 

Intuitively, a strategy for a player P of a game G specifies a choice (left or right) for every node of 
G at which it is P's turn to move. A strategy profile specifies a strategy for every player. Following 
[8], we define the set of strategy profiles directly, as the final coalgebra (<S, a) of the functor 

F SP : X h-> U + AxC x X x X. 

There is an evident natural transformation t : F$p — > Fq defined by projection, which induces a 
functor from the category of Fsp-coalgebras to the category of Fc-coalgebras. It sends a strategy 
profile s e <S to the underlying game tree game(s) G G- We say that s is a strategy profile for the 
game G if G = game(s). 

We now define a number of strategy profiles for the 0/1 and dollar auction games. 



3.1 The 0/1 game 

We define two strategy profiles for the 0/1 game where A always stops (plays left) and B always 
continues (plays right), by the following simultaneous corecursion: 

AsBc = (A, I, v, BcAs) 
BcAs = (B, r, w, AsBc) 

We can depict this strategy profile as follows: 

r r f , A r 



Formally, the strategy profile is given by [AsBc], the image of the strategy profile under the 
morphism to the final coalgebra. The strategy profile [AcBs], where A always continues and B 
always stops, is defined symmetrically. 
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3.2 The dollar auction 



In a similar fashion, we define strategy profiles for the dollar auction by simultaneous corecursion 
on an infinite family of variables {AsBc„, BcAs„ | n G N}: 

AsBc„ = (A, I, v n , BcAs„) 
BcAs n = (B, r, w n , AsBc n+ i). 

Formally, JAsBco] is a strategy profile for the dollar auction game in which A always stops and B 
always continues. The strategy profile [AcBso] where A always continues and B always stops is 
defined symmetrically. 

4 Subgame perfect equilibrium 

We now show, following [8], how the game-theoretic notion of subgame perfect equilibrium 
can be defined coalgebraically. Firstly, we introduce two auxiliary notions. 

4.1 Weak and strong convergence 

We introduce two predicates on strategy profiles, of weak and strong convergence. A strategy 
profile is weakly convergent if following the choices that it specifies from the root eventually leads 
to a leaf; it is strongly convergent if this holds in every sub-profile. 
We define weak convergence by: 

4 (s = U) V (s = (P,\,s h s r ) A V (s = {P,r,s u s r } A s r |). 

More formally, weak convergence is an element of the powerset V(S). It is defined as the least 
fixpoint of the monotone function 

.U : V{S) — ► V{S) :: S ^ UU {(P,\,a\,a r ) \ s\ G S} U {(P,r,s u s r ) \ s r G S}. 

Strong convergence is defined as follows: 

4 <^ (s = U) V (s = (P,c,s\,s r ) A 4 A s,J| A s r JJ.). 

More formally, it is the greatest fixpoint of the monotone function defined analogously to f± 

U ■■ V(S) — > V{S) :: S ^ U U {(P,l,s,, Sr ) | s, e S, s r G S}. 

These fixpoints exist by the Knaster-Tarski fixed point theorem [51 116j . 

Note that weak convergence is an inductive notion; the profile must specify a finite path from 
the root to a leaf. Thus it is defined as a least fixpoint. Strong convergence, by contrast, expresses 
a constraint on all subtrees of an infinite tree, and hence is defined coinductively, as a greatest 
fixpoint. The examples of the strategy profiles described in the previous section are all strongly 
convergent. 

The relationship between the two notions can be nicely characterized in terms of coalgebraic 
modal logic [T31 [T2] . We can define an 'always' modality □ as an operator on V(S): 

s\=n<j) = s \= A s= (P,c,s\,s r ) => si |= U(f> A s r \= □</>. 

This operator is defined coinductively as a greatest fixpoint. This modality generalizes straighfor- 
wardly to any polynomial functor, and can in fact be defined in a much more general way in the 
context of coalgebraic modal logic [13j [12]. If we write WC for the weak convergence predicate, 
and SC for the strong convergence predicate, we have the following: 

Proposition 4.1 SC = DWC. 
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4.2 Utility functions induced by strategy profiles 

A strategy profile s induces a utility function s. This function is denned as follows: 







s = U 


H 




S = (P,\,S\,S r ) 




[ S r , 


s = (P,r,s\,s r ) 



In general, this function may be partial; however, if s is weakly convergent, s is always a well- 
defined total function in U. 

4.3 Subgame perfect equilibria 

We are now ready to define the notion of a strategy profile being a subgame perfect equilibrium. 
Firstly, we define a predicate PE on strategy profiles: 

PE(s) S$ A { S = (P,\,8l,S T ) => S|(P)>S r (P)) A { S = (P, T , S|, S r ) => S r {P) > S|(P)). 

This predicate is defined explicitly in terms of previous notions; neither induction nor coinduction 
is used. It says that the choice at the root for player P results in a better payoff for P than 
the other choice would have done. Note that this predicate implies in particular that strong 
convergence holds. 

We now define the subgame perfect equilibrium predicate SPE on 5, coinductively as a greatest 
fixpoint. 

SPE(s) <s=> PE(s) A (s = (P,c,s\,s r ) => SPE(sj) A SPE(s r )). 

This says that the PE predicate holds at every node of the tree and we get the following analogue 
of Proposition 14. II 

Proposition 4.2 SPE = DPE. 

The next step is to show that all the strategy profiles discussed in the previous section are in 
fact subgame perfect equilibria. In order to do this, we will need to have an appropriate proof 
principle in place. This is the topic of the next section. 

5 Predicate coinduction 

The main emphasis in coinductive proofs has been on proving equations; the main tool for this 
is provided by the notion of bisimulation. However, as emphasized by Kozen [7], the scope of 
coinductive methods is broader than this. In our case, we are interested in predicates (properties) 
rather than equations. In particular, we wish to show that various elements of the final coalgebra 
S satisfy the SPE predicate. 

We formulate a proof principle which is adequate to carry out these proofs, and justify it in 
terms of Kozen's metric coinduction principle. It should be possible to give a much more general 
account; this is an interesting challenge, which is left to future work. 

We formulate the principle in terms of an arbitrary polynomial functor T and apply it in the 
case T = F$p. The final coalgebra of T is denoted (<S, a). 

Firstly, we need some notation. Suppose we have a T-coalgebra (X, a), which we think of as 
a corecursive system of equations on the set of variables X. We define a map 

a : S x — > S x :: r; H> [x i-> cr _1 o Tr\ o a(x) ]. 

Here cr _1 : TS — > S is the inverse of cr, which is an isomorphism by the Lambek lemma. 

The following proposition follows directly by unravelling the definitions and applying the final 
coalgebra property: 
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Proposition 5.1 The map a has a unique fixpoint a* £ S x ; moreover, a* — [•], the unique 
coalgebra morphism from (X, a) to the final coalgebra. 

Now let C 5 be a predicate on iS. We lift this predicate to 4> x C S x : 

rj £ <j) x ^=^> Va; £ X. rj(x) £ 4>. 

We say that <j> is an invariant if <j> = Di/j for some predicate ijj and formulate the predicate 
coinduction principle as a proof rule: 

4> invariant, 0=^0, r\ £ <f> x => oc{rj) £ (f> x 
a* £(j) x 

Using Proposition [ITTJ we can restate the conclusion as follows: 

Vx £ X. \x\ £ (/). 

In order to show the soundness of this principle, we reduce it to the following metric coinduction 
principle due to Kozen 70 

Proposition 5.2 Let M be a complete metric space, u : M — >• M a contractive map, and C C M 
a non-empty closed subset of M . We write u* for the unique fixpoint of u, which exists by the 
Banach fixpoint theorem. Then the following proof rule is valid: 

x £ C u(x) £ C 
u* £ C ■ 

Proof We have u* = lim^oo u n (a) for any a £ M . Since C is non-empty, we can take a £ C. 
By the premise of the rule, u n (a) £ C for all n. Since C is closed, u* £ C. □ 

The reduction is an immediate consequence of the following result. 

Proposition 5.3 We can define a distance function d : S 2 — > [0, 1] such that the following holds: 

1. (S,d) is a complete ultrametric space. 

2. An invariant (f> C S is closed in this space. 

3. For any set X , (S x ,d ) is a complete ultrametric space, where 

d x (rj,n) := sup d(r) x ,fj, x ). 
xex 

4- For any coalgebra (X, a), the map a is contractive. 

Proof This is essentially a variation of well-known results, see e.g. Q]. The basic point is that 
T is cocontinuous, and hence the final T-coalgebra is the limit of the w° p -diagram 

1 Tl T 2 1 • • • T k l • • • (1) 

where 1 is the terminal object in Set. The connecting maps are T k l : T k+1 l — > T 1, where 
! : Tl — > 1 is the unique map to the terminal object. Starting at the unique map ttq to the 
terminal object, we define the maps TTfe : S — > T fc l inductively by 

7rfc+1 ;= S — TS — TT fc l. 
Then (S, {^k}) is the limit cone of the diagram ([T]). This has the following consequences: 
• For all x,y £ S: 

x = y Vfc. ir k x = Ti k y. 

1 Kozen states a more general principle, but this version will be sufficient for our purposes. 
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• Given a sequence {xk} with Xk £ T k l and Xk = T k \(xk+i), there is a unique x £ S such 
that for all k, Xk = 7tk%- 



The ultramctric on S is given by 
d(x,y) = 



0, x = y 

2~ k , least fc such that ir^x ^ 7rfcy, x ^ y. 



The properties stated in the proposition follow readily from this description. For example, the 
completeness of d x follows from the fact that Cauchy convergence in this metric implies that for 
each k, for some N, for all x £ X, nk(r]j(x)) is fixed for all j > N. This implies a uniform mode 
of convergence, and thus justifies taking limits pointwise in S x . □ 



6 Proving subgame perfect equilibria 

We now apply predicate coinduction to show that the strategy profiles defined in Section [3] are 
subgame perfect equilibria. 

We begin with the strategy profiles for the 0/1-game. 

Proposition 6.1 The strategy profiles [AcBs] and JAsBcJ are subgame perfect equilibria. 

Proof Applying predicate coinduction, to show that [AsBc] is SPE, we must show that (A, I, v, BcAs) 
and (B, r, w, AsBc) are SPE, under the assumption that AsBc and BcAs are SPE. Using the coin- 
duction hypothesis, we just need to show that these nodes are PE. Computing their induced payoff 
functions explicitly, this reduces to verifying the inequalities 

v(A) = > = v(A), v(B) = 1 > = w(B). 

The verification that JAcBsJ is SPE is similar. □ 
We now turn to the dollar auction. We recall the parameter r used to define the utility functions 
v n , w n . 

Proposition 6.2 If r > 1, the strategy profiles [AcBs„J and [AsBc„J are subgame perfect equi- 
libria for all n. 

Proof The coinduction hypothesis is that AsBc„ and BcAs„ are SPE for all n. We must show 
that (A, \,v n , BcAs„) and (B, r, w n , AsBc n +i) are SPE for all n. In similar fashion to the previous 
result, this reduces to verifying the inequalities 

v n (A) = -n> -(n + 1) = v n+ i(A), v n+1 (B) = r-(n + l)>-n = w n {B). 

The latter inequalities are satisfied if and only if r > 1. □ 



7 The one-deviation principle 

We now verify an important property of subgame perfect equilibria: a strategy profile is SPE if and 
only if it dominates any profile which differs from it in exactly one choice. In the standard game 
theoretical literature [3] , this is proved for infinite games only under strong continuity assumptions 
on the payoffs, which amount to discounting beyond some finite horizon, thus allowing reduction 
to standard backwards induction reasoning. We need no such assumptions. 

Given a strategy profile s of the form (P, c, s\, s r ), we say that a profile t for the same game is 
a one-deviation from s if t has one of the following forms: 

• (P, c' , s\, s r ), where d ^ c. 

• (P, c, s[, s r ), where sf is a one-deviation from s\. 
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• (P, c, S|, s' r ), where s' r is a one-deviation from s r . 

This is an inductive definition. Given a strategy profile s we define a relation s )^ t, where t is a 
one-deviation of s, inductively as follows: 

• If s = (P, I, si, s r > and t = (P, r, s\, s r ), then s M iff S|(P) > §r (•?))< 

• If s = (P,r,si,s r ) and i = (P, l,S|,s r ), then s ^= * iff s r (P) > S|(P)). 

• If s = (P, c, si, s r ) and i = (P, c, s[, s r ), then s >p t iff si fc= s(. 

• If s = (P, c, si, s r ) and i = (P, c, si, sf), then s ^= i iff s r ^= s' r . 

Proposition 7.1 (The one-deviation principle) ^4 strongly convergent strategy profile s is 
SPE if and only if for every one-deviation t, s )p t. 

Proof Firstly, note that if -i(s )p t) for some one-deviation t, this means that some subprofile 
of s docs not satisfy PE, and since SPE = dPE, this implies that s does not satisfy SPE. For the 
converse, note that if for some one-deviation t, s ^= t, this implies that the sub-profile of s whose 
root is at the node where t differs from s satisfies PE. If this holds for all one-deviations, then all 
sub-profiles of s satisfy PE, and hence s satisfies SPE. □ 

8 Complete Characterization of SPE for the Dollar Auction 

We can give a complete characterization of the subgame perfect equilibria for the dollar auction 
game. 

Theorem 8.1 A strategy profile for the dollar auction game is SPE if and only if it is of the 
form AsBc or A always stops and B always continues or symmetrically AcBs or B always stops 
and A always continues. 

Proof Firstly, note that if both players always continue from some point in the game, the profile 
will not be strongly convergent. Now suppose that both players choose to stop at some nodes of 
the game. Then there must be some node v where player a chooses to stop, such that the next 
player choosing to stop is the other player. But then v is not in PE, since a could improve his 
payoff from that node by choosing to continue. Thus one player must always continue in any SPE 
profile, while the other player a must stop infinitely often. Finally, a must in fact always choose to 
stop, since otherwise he could improve his payoff from any node where he chooses to continue. □ 

This analysis applies to any game sharing the following features of the dollar auction game: 

1. At any point of the game, it is always better for a given player if the other player stops first. 

2. At any point of the game, it is (strictly) better for the player who is the first to stop from 
that point to stop immediately rather than later. 

As a final remark on the dollar auction game, we note that Lescanne proposes to use this 
form of analysis to explain the rationality of infinite escalation [8]. If we don't know which 
strategy the other player is following, we always have an incentive to continue! However, by our 
characterization result, after one round where both players choose to continue, they both know 
they are not in an SPE — and all bets are off! Thus it seems to us that a comprehensive 
analysis of escalation should use a refined model with an explicit representation of the beliefs of 
the players, as in Harsanyi type spaces [4] — which can also be modelled coalgebraically [11] . 
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9 Further Directions 



The present paper contains what should be regarded as some very preliminary results, presented in 
a manner which is mainly aimed at computer scientists and mathematicians rather than economists 
and game theorists. Nevertheless, in our view the general idea of applying coalgebraic and other 
structural methods which have been developed in computer science to economics and game theory 
is promising, and deserves further study and development. In particular, many topics in economics 
which refer to infinite horizons and rcflcxivity seem tailor-made for the use of coalgebraic methods. 
At the same time, they can suggest new challenges and technical directions for coalgebra. 
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